Papers with self-contrastive learning
Efficient Document Embeddings via Self-Contrastive Bregman Divergence Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Despite recent advances in transformer-based sentence encoders, the encoding of long documents (Ks of words) is still challenging with respect to both efficiency and quality considerations. |
| Approach: | They propose to combine a self-contrastive siamese network and a convex neural Bregman divergence network to train longfomer-based document encoders using an unsupervised contrastive learning method. |
| Outcome: | The proposed model outperforms baseline models on three long document topic classification tasks from the legal and biomedical domains. |